Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

internal: check for new majors in v2 #3078

Open
wants to merge 14 commits into
base: v2-dev
Choose a base branch
from

Conversation

quinna-h
Copy link
Contributor

@quinna-h quinna-h commented Jan 10, 2025

What does this PR do?

Checks for new major versions on github in v2

Motivation

In the original PR, we used a hardcoded, manual integration_go.mod to detect if new major versions (corresponding to new packages) have been released. However, in v2 each package will be treated as its own contrib with its own corresponding go.mod. We want to parse these go.mod files to check the current latest major, and output if a new major is available.

  1. Check the latest major we support for a given package / module (parsing the go.mod files).

  2. Fetch / check for new major versions from github:

  • Run go list -m -json <repository>@<version> and extract Origin[url]
  • List all tags with git ls-remote --tags <repository>
  • Extract all majors, get the latest version of each major. For each latest version of each major: curl https://raw.githubusercontent.com/<module>/refs/tags/<latest_version>/go.mod
    • If the request returns 404, the module is not a go module and we can skip it
    • Otherwise, we parse the modfile and extract the mod name
  1. Compare the results from 1 (latest contrib major which we support) and 2 (latest major on github). If 2 is greater than 1 and there is a new module for the new major, output "New latest major on Github: {github_latest_major}"

Example output from workflow run: https://github.com/DataDog/dd-trace-go/actions/runs/12952678774/job/36130554853

TODO:

  • Check if need to add or update packages in instrumentation/packages.go

Reviewer's Checklist

  • Changed code has unit tests for its functionality at or near 100% coverage.
  • System-Tests covering this feature have been added and enabled with the va.b.c-dev version tag.
  • There is a benchmark for any new code, or changes to existing code.
  • If this interacts with the agent in a new way, a system test has been added.
  • Add an appropriate team label so this PR gets put in the right place for the release notes.
  • Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild.
  • For internal contributors, a matching PR should be created to the v2-dev branch and reviewed by @DataDog/apm-go.

Unsure? Have a question? Request a review!

@datadog-datadog-prod-us1
Copy link

datadog-datadog-prod-us1 bot commented Jan 10, 2025

Datadog Report

Branch report: quinna.halim/v2-output-new-major-versions
Commit report: 4cc7e3d
Test service: dd-trace-go

✅ 0 Failed, 4088 Passed, 64 Skipped, 2m 39.44s Total Time

@pr-commenter
Copy link

pr-commenter bot commented Jan 10, 2025

Benchmarks

Benchmark execution time: 2025-01-24 18:51:49

Comparing candidate commit 46f943c in PR branch quinna.halim/v2-output-new-major-versions with baseline commit 89bfc59 in branch v2-dev.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 51 metrics, 2 unstable metrics.

new_latest_majors.go Outdated Show resolved Hide resolved
@quinna-h quinna-h requested a review from rarguelloF January 10, 2025 18:36
@quinna-h quinna-h marked this pull request as ready for review January 13, 2025 18:32
@quinna-h quinna-h requested review from a team as code owners January 13, 2025 18:32
return "", fmt.Errorf("no valid versions found")
}

// // walk through contrib directory
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO(quinna): remove these before merge.
leaving comments for clarity for reviewers.

@@ -804,6 +804,10 @@ var packages = map[Package]PackageInfo{
},
}

func GetPackages() map[Package]PackageInfo {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: remove or change this, depending on which PR gets merged first

.github/workflows/apps/outdated_github_majors.go Outdated Show resolved Hide resolved
.github/workflows/apps/outdated_github_majors.go Outdated Show resolved Hide resolved
@quinna-h quinna-h marked this pull request as draft January 22, 2025 14:32
Comment on lines 35 to 40
var stdLibPackages = map[string]struct{}{
"log/slog": {},
"os": {},
"net/http": {},
"database/sql": {},
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is redeclared from https://github.com/DataDog/dd-trace-go/pull/3070/files#diff-1196dd5bce1d4f0ed73101590ba38f17605ce38d6c18081f4bdaa2a397ab5adeR72-R77
wonder if we can just put this in instrumentation/packages.go and export it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: would the standard in Go here be:

  1. declare this as a public variable such asStandardPackages in instrumentation/packages.go?
  2. create a public method such as GetStandardPackage in instrumentation/packages.go?

Copy link
Contributor Author

@quinna-h quinna-h Jan 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made this StandardPackages in packages.go; but alternatively, could create a new field in PackageInfo

@quinna-h quinna-h marked this pull request as ready for review January 23, 2025 15:10
@quinna-h quinna-h requested a review from bouwkast January 23, 2025 15:56
Copy link
Contributor

@bouwkast bouwkast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Overall looks good to me, just a few comments

.github/workflows/apps/new_latest_major_versions.go Outdated Show resolved Hide resolved
.github/workflows/apps/new_latest_major_versions.go Outdated Show resolved Hide resolved

// Fetch `go.mod` with command
// curl https://raw.githubusercontent.com/<module>/refs/tags/<latest>/go.mod
goModURL := fmt.Sprintf("https://raw.githubusercontent.com/%s/refs/tags/%s/go.mod", base, latest)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was just testing this out with other files

It looks like https://raw.githubusercontent.com/OWNER/REPO/TAG/FILE also works

Just wondering if there is any difference, not really seeing anything online though for it

@datadog-datadog-prod-us1
Copy link

datadog-datadog-prod-us1 bot commented Jan 24, 2025

Datadog Report

Branch report: quinna.halim/v2-output-new-major-versions
Commit report: 469778d
Test service: dd-trace-go

✅ 0 Failed, 4095 Passed, 64 Skipped, 2m 43.83s Total Time

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: not sure why these packages are showing as changes after rebasing off v2-dev

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants